A Dimensionality Reduction Technique for Collaborative Filtering

Recommender systems make suggestions about products or services based on matching known or estimated preferences of users with properties of products or services (contentbased), properties of other users considered to be similar (collaborative filtering), or some hybrid approach. Collaborative filtering is widely used in E-commerce. To generate accurate recommendations in collaborative filtering, the properties of a new user must be matched with those of existing users as accurately as possible. The available data is very large, and the matching must be computed in real time. Existing heuristics are quite ineffective. We introduce novel algorithms that use “positive” nearest-neighbor matching, that is they find neighbors whose attribute values exceed those of the new user. The algorithms use singular value decomposition as a dimension-reduction technique, and match in a collection of lower-dimensional spaces. Although singular value decomposition is an obvious approach to dimension reduction, it requires some care to work effectively in this setting. Performance and quality of recommendations are measured using a movie database. We show that reasonable matches can be found in time O(m log n), using O(nm) storage space, where n is the number of users and m the number of attributes or products for which users may express preferences. This is in contrast to “approximate nearest neighbor” techniques that require either time or storage exponential in m.